Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add T4 dataset updater with the specifeid FastLabel annotations #172

Merged
merged 14 commits into from
Jan 2, 2025

Conversation

ktro2828
Copy link
Contributor

@ktro2828 ktro2828 commented Nov 27, 2024

Description

Add support of updating the existing T4 dataset with a new FastLabel annotation.

AnnotationFilesUpdater parse annotation file(s) and insert the corresponding annotation information to the scene.

Sample configuration file:

task: update_t4_with_fastlabel  # Task Name
description:
  visibility:
    full: "No occlusion of the object."
    ...
  camera_index:
    CAM_FRONT: 0
    ...

conversion:
  input_base: ./data/input_t4_format # could be non_annotated_t4_format or t4_format_3d_annotated
  input_anno_base: ./data/fastlabel
  output_base: ./data/output_t4_format # currently, this only includes the 2D annotations
  dataset_corresponding:
    # input t4dataset_name: FastLabel json file name
    T4DatasetName: FastLabelAnnotationFile

How to review

How to test

test data

test command

python3 -m perception_dataset.convert --config ./configs/update_wt_with_fastlabel_sample.yaml

Reference

Notes for reviewer

@ktro2828 ktro2828 force-pushed the feat/update-dataset-with-fastlabel branch 3 times, most recently from 5262416 to 650bdf3 Compare November 27, 2024 11:07
@ktro2828 ktro2828 force-pushed the feat/update-dataset-with-fastlabel branch from d96f426 to 4cf260a Compare November 27, 2024 22:32
@ktro2828 ktro2828 marked this pull request as ready for review November 27, 2024 22:34
@ktro2828 ktro2828 requested a review from miursh November 27, 2024 22:34
@ktro2828 ktro2828 force-pushed the feat/update-dataset-with-fastlabel branch from 7082f31 to 547af26 Compare November 27, 2024 23:07
Signed-off-by: ktro2828 <[email protected]>
@ktro2828 ktro2828 force-pushed the feat/update-dataset-with-fastlabel branch from 547af26 to bc5c3fe Compare November 27, 2024 23:10
* update configs

Signed-off-by: Shunsuke Miura <[email protected]>

* add label converter to FastLabel2dToT4Converter

Signed-off-by: Shunsuke Miura <[email protected]>

* fix bug in from_json of surface_ann

Signed-off-by: Shunsuke Miura <[email protected]>

* add automatic_annotation key to sample_annotation/object_ann/surface_ann

Signed-off-by: Shunsuke Miura <[email protected]>

* accelerate default object_mask generation

Signed-off-by: Shunsuke Miura <[email protected]>

* feat: improve T4 updater functionality and directory handling

Signed-off-by: Shunsuke Miura <[email protected]>

* feat: support polygon-to-bbox conversion and improve polygon-to-RLE conversion

* update object config

Signed-off-by: Shunsuke Miura <[email protected]>

* feat: update annotation loader

* feat: only convert to bbox for objects labels

Signed-off-by: Shunsuke Miura <[email protected]>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* fix: handling automatic_annotation entry

Signed-off-by: Shunsuke Miura <[email protected]>

* fix: bug fix in rle_from_points

Signed-off-by: Shunsuke Miura <[email protected]>

* fix: add default value for automatic_annotation

Signed-off-by: Shunsuke Miura <[email protected]>

---------

Signed-off-by: Shunsuke Miura <[email protected]>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
@ktro2828
Copy link
Contributor Author

ktro2828 commented Dec 18, 2024

miursh and others added 10 commits December 19, 2024 09:51
…nd DeepenToT4Converter (#180)

* feat: enhance surface category handling in AnnotationFilesGenerator and DeepenToT4Converter

Signed-off-by: Shunsuke Miura <[email protected]>

* delete unnecessary import

Signed-off-by: Shunsuke Miura <[email protected]>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Signed-off-by: Shunsuke Miura <[email protected]>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* feat: enhance surface category handling in AnnotationFilesGenerator and DeepenToT4Converter

Signed-off-by: Shunsuke Miura <[email protected]>

# Conflicts:
#	perception_dataset/deepen/deepen_to_t4_converter.py
#	perception_dataset/fastlabel_to_t4/fastlabel_2d_to_t4_converter.py
#	perception_dataset/t4_dataset/annotation_files_generator.py

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* refactor: parallelizate fl annotation processing

Signed-off-by: Shunsuke Miura <[email protected]>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Signed-off-by: Shunsuke Miura <[email protected]>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
# Conflicts:
#	perception_dataset/t4_dataset/classes/object_ann.py
#	perception_dataset/t4_dataset/classes/surface_ann.py
* merge _load_annotation_jsons function to one

Signed-off-by: Shunsuke Miura <[email protected]>

* add default value for automatic_annotation

Signed-off-by: Shunsuke Miura <[email protected]>

* change to non-keyframe for no-annotation frame

Signed-off-by: Shunsuke Miura <[email protected]>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* remove unused import

Signed-off-by: Shunsuke Miura <[email protected]>

---------

Signed-off-by: Shunsuke Miura <[email protected]>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Signed-off-by: Shunsuke Miura <[email protected]>
Signed-off-by: Shunsuke Miura <[email protected]>
Copy link
Collaborator

@miursh miursh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@ktro2828 ktro2828 merged commit 93f763b into main Jan 2, 2025
5 checks passed
@ktro2828 ktro2828 deleted the feat/update-dataset-with-fastlabel branch January 2, 2025 10:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants